14 research outputs found
Robust estimation of exposure ratios in multi-exposure image stacks
Merging multi-exposure image stacks into a high dynamic range (HDR) image
requires knowledge of accurate exposure times. When exposure times are
inaccurate, for example, when they are extracted from a camera's EXIF metadata,
the reconstructed HDR images reveal banding artifacts at smooth gradients. To
remedy this, we propose to estimate exposure ratios directly from the input
images. We derive the exposure time estimation as an optimization problem, in
which pixels are selected from pairs of exposures to minimize estimation error
caused by camera noise. When pixel values are represented in the logarithmic
domain, the problem can be solved efficiently using a linear solver. We
demonstrate that the estimation can be easily made robust to pixel misalignment
caused by camera or object motion by collecting pixels from multiple spatial
tiles. The proposed automatic exposure estimation and alignment eliminates
banding artifacts in popular datasets and is essential for applications that
require physically accurate reconstructions, such as measuring the modulation
transfer function of a display. The code for the method is available.Comment: 11 pages, 11 figures, journa
Recommended from our members
Noise-Aware Merging of High Dynamic Range Image Stacks Without Camera Calibration
A near-optimal reconstruction of the radiance of a High Dynamic Range scene
from an exposure stack can be obtained by modeling the camera noise
distribution. The latent radiance is then estimated using Maximum Likelihood
Estimation. But this requires a well-calibrated noise model of the camera,
which is difficult to obtain in practice. We show that an unbiased estimation
of comparable variance can be obtained with a simpler Poisson noise estimator,
which does not require the knowledge of camera-specific noise parameters. We
demonstrate this empirically for four different cameras, ranging from a
smartphone camera to a full-frame mirrorless camera. Our experimental results
are consistent for simulated as well as real images, and across different
camera settings
Distilling Style from Image Pairs for Global Forward and Inverse Tone Mapping
Many image enhancement or editing operations, such as forward and inverse
tone mapping or color grading, do not have a unique solution, but instead a
range of solutions, each representing a different style. Despite this, existing
learning-based methods attempt to learn a unique mapping, disregarding this
style. In this work, we show that information about the style can be distilled
from collections of image pairs and encoded into a 2- or 3-dimensional vector.
This gives us not only an efficient representation but also an interpretable
latent space for editing the image style. We represent the global color mapping
between a pair of images as a custom normalizing flow, conditioned on a
polynomial basis of the pixel color. We show that such a network is more
effective than PCA or VAE at encoding image style in low-dimensional space and
lets us obtain an accuracy close to 40 dB, which is about 7-10 dB improvement
over the state-of-the-art methods.Comment: Published in European Conference on Visual Media Production (CVMP
'22
HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content
High-Dynamic-Range Visual-Difference-Predictor version 3, or HDR-VDP-3, is a
visual metric that can fulfill several tasks, such as full-reference
image/video quality assessment, prediction of visual differences between a pair
of images, or prediction of contrast distortions. Here we present a high-level
overview of the metric, position it with respect to related work, explain the
main differences compared to version 2.2, and describe how the metric was
adapted for the HDR Video Quality Measurement Grand Challenge 2023
Perceptual Quality Assessment of NeRF and Neural View Synthesis Methods for Front-Facing Views
Neural view synthesis (NVS) is one of the most successful techniques for
synthesizing free viewpoint videos, capable of achieving high fidelity from
only a sparse set of captured images. This success has led to many variants of
the techniques, each evaluated on a set of test views typically using image
quality metrics such as PSNR, SSIM, or LPIPS. There has been a lack of research
on how NVS methods perform with respect to perceived video quality. We present
the first study on perceptual evaluation of NVS and NeRF variants. For this
study, we collected two datasets of scenes captured in a controlled lab
environment as well as in-the-wild. In contrast to existing datasets, these
scenes come with reference video sequences, allowing us to test for temporal
artifacts and subtle distortions that are easily overlooked when viewing only
static images. We measured the quality of videos synthesized by several NVS
methods in a well-controlled perceptual quality assessment experiment as well
as with many existing state-of-the-art image/video quality metrics. We present
a detailed analysis of the results and recommendations for dataset and metric
selection for NVS evaluation
Neural Fields with Hard Constraints of Arbitrary Differential Order
While deep learning techniques have become extremely popular for solving a
broad range of optimization problems, methods to enforce hard constraints
during optimization, particularly on deep neural networks, remain
underdeveloped. Inspired by the rich literature on meshless interpolation and
its extension to spectral collocation methods in scientific computing, we
develop a series of approaches for enforcing hard constraints on neural fields,
which we refer to as Constrained Neural Fields (CNF). The constraints can be
specified as a linear operator applied to the neural field and its derivatives.
We also design specific model representations and training strategies for
problems where standard models may encounter difficulties, such as conditioning
of the system, memory consumption, and capacity of the network when being
constrained. Our approaches are demonstrated in a wide range of real-world
applications. Additionally, we develop a framework that enables highly
efficient model and constraint specification, which can be readily applied to
any downstream task where hard constraints need to be explicitly satisfied
during optimization.Comment: 37th Conference on Neural Information Processing Systems (NeurIPS
2023
Recommended from our members
Comparison of single image HDR reconstruction methods — the caveats of quality assessment
As the problem of reconstructing high dynamic range (HDR) images from a single exposure has attracted much research effort, it is essential to provide a robust protocol and clear guidelines on how to evaluate and compare new methods. In this work, we compared six recent single image HDR reconstruction (SI-HDR) methods in a subjective image quality experiment on an HDR display. We found that only two methods produced results that are, on average, more preferred than the unprocessed single exposure images. When the same methods are evaluated using image quality metrics, as typically done in papers, the metric predictions correlate poorly with subjective quality scores. The main reason is a significant tone and color difference between the reference and reconstructed HDR images. To improve the predictions of image quality metrics, we propose correcting for the inaccuracies of the estimated camera response curve before computing quality values. We further analyze the sources of prediction noise when evaluating SI-HDR methods and demonstrate that existing metrics can reliably predict only large quality differences
Recommended from our members
Dataset for HDR4CV: High dynamic range dataset with adversarial illumination for testing computer vision methods
This dataset is intended to test the robustness of computer vision methods under challenging illumination conditions. The dataset includes four scenes (*Faces*, *Tunnel*, *Street-parallel*, *Street-diagonal*), which were captured multiple times, each time under a different illumination condition (**night**, **hdr**, **glare**, and **uniform**). A non-challenging **uniform** illumination condition serves as a reference and can be used to generate labels for the other three challenging illumination conditions and test a given computer vision method. This lets us avoid the tedious task of manually labeling the data.
Refer to README.md for further details